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Introduction 

In this age of heightened accountability, academia is increasingly being asked 
to link assessment to candidate performance outcomes in multiple ways. Research 
demonstrates the importance of aligning assessment with content standards but 
cautions that it is critical that assessments match the content, cover a wide range 
of knowledge, are cognitively demanding, and avoid irrelevant materials (AERA, 
2003). Case-based pedagogy is one way to link program content to classroom 
practice. Much of the research on case methods calls for the use of cases to “create 
bridges across the great chasm that divides policy from practice” (Shulman, 2000, 

p. 2) in order to help teachers understand how practice 
is constructed in the classroom. Within ease-based 
pedagogy, the cases become teaching tools that serve 
as a context for making meaning of concepts presented 
during instruction in a variety of instructional settings, 
and thus make understanding transparent. 

In this article we examine the use of candidate- 
authored case studies as a culminating assessment 
activity in one Reading and Language Arts Specialist 
Credential Program (RRLA) in a large, urban public 
university with a diverse student population in south- 
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ern California. Candidates in the program earn an M.A. in Education, Option in 
Reading, along with the state-issued specialist credential. We ask the following 
research questions in order to examine the use of case-based assessment in the 
RRLA program: 

1 . Are case studies an effective way for graduate candidates to demonstrate 
knowledge and skills learned in the program? 

2. Can candidates use case studies to demonstrate what they know about 
serving culturally and linguistically diverse urban students? 

Theoretical Framework 

Much has been written about how assessment practices have and have not changed 
in the past 100 years (Brown, 1996; Shepard, 2004). While experts argue that there is 
reciprocity between assessment and instruction, the construction of school practice 
is often informed by outdated theories that do not consider new understandings from 
the field (Brown, 1996), frequently creating a disconnect between assessment and 
instruction. Shepard (2004) argues that “the content of assessments should match 
challenging subject matter standards and serve to instantiate what it means to know 
and learn in each of the disciplines” (p. 1 62 1 ) and she proposes a social-constructivist 
concept of assessment where dynamic, ongoing assessment offers candidates explicit 
evaluation criteria, in addition to support and assistance as feedback as they progress 
through the program. Thus the instructional program leads candidates towards the 
desired competencies that the assessment seeks to measure. 

The use of case-based methodology to measure teacher competence is one 
way to link assessment to practice (Shulman, 2002) and to shift from traditional 
modes of evaluation into a more dynamic and authentic review of learning. While 
the professional fields of business and law have used cases and case methods for 
years, the idea of using them in education has emerged over the last 1 5 years as a 
promising idea. Merseth (1991) argues that the current move toward a case-based 
pedagogy is due to a growing interest in teacher knowledge and cognition as well 
as an acknowledgement of the complexities of teaching. 

With this growing awareness, the efforts to define case studies have gained 
prominence. According to Merseth ( 1 994) a case is “a descriptive narrative document 
that is based on a real life situation or event” (p. 2). The author further describes 
the case as having three essential elements: a firm base in reality, a reliance on 
research, and the development of multiple perspectives by those who use them. 
Complicating these efforts are the various uses of case studies. For example, cases 
can be used as stimulants for reflection, techniques to enrich field experiences, and 
tools for professional evaluation. They can further serve to frame conversations 
between mentors and novices, orient individuals to particular ways of thinking and 
initiate discussions from different perspectives (Merseth). Cases methods may be 
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used during large- and small-group discussions, role-playing, written analysis or 
team-based discussion (Merseth). Additionally, the use of cases for assessment can 
help teachers to reflect on their own practice and make more informed decisions 
regarding that practice (Harrington, 1995; Merseth, 1996; Shulman, 1987). 

Why the growing interest in case-based methodology? What do cases or case 
studies offer that differs from other, more traditional research methods? While 
opinions may vary, the fundamental responses to these questions include context 
and authenticity. Not only do the cases themselves provide compelling contexts 
for discussing and making sense of classroom practice, but the notion that they 
are authentic representations of such practice also supports their use. Authenticity 
enhances the effectiveness of cases by adding context to theory (Colbert, Trimble 
& Desberg, 1 996). The paramount concern is to make cases real and to use true-life 
stories that illustrate key educational theories and bring up issues that are critical 
to the professional growth of teachers. In this way cases can help students think 
productively about concrete experiences and enhance the ability to learn from 
their own experiences as they employ theoretical concepts to illuminate a practice 
problem or situation and practice how to think professionally about real problems 
and situations (Kleinfeld, 1996). 

While much can be related to the use of case-based pedagogy as a teaching tool 
where students read and respond to cases, its use as a form of assessment where 
students create their own cases as a means of demonstrating competencies is rela- 
tively new. When using case studies for assessment it is important to consider not 
only the benefits but also the drawbacks. The benefits of utilizing case studies in 
instruction include the way that cases model how to think professionally about real 
problems and situations, helping candidates to think productively about concrete 
experiences (Kleinfeld, 1990). When cases are used for assessment, candidates can 
be presented with situations that require them to apply their knowledge and skills 
to solve real problems in the field. Tellis (1994) suggests that while case studies 
are dynamic, authentic, contextualized and linked to reality, they also present the 
candidate with challenges in terms of time expended, bias, objectivity/subjectivity, 
and other obstacles that arise, including blocked access and inability to build rap- 
port with the case study student. Furthermore, even though the use of case studies 
as a means of authentic assessment linking theory to practice leans toward a more 
qualitative approach to assessment, current and national trends in education favor 
more quantitative systematic approaches. 


Context for the Study 

The Reading and Language Arts Specialist Credential is a state-authorized 
advanced credential issued to teachers who have a minimum of three years of 
teaching experience. Courses in the RRLA, which is accredited by the California 
Commission on Teacher Credentialing (CCTC) and the National Council for Ac- 
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creditation of Teacher Education (NCATE), also meet the standards of the Interna- 
tional Reading Association. Program standards emphasize foundational knowledge, 
instructional strategies and curriculum materials, assessment, diagnosis and evalua- 
tion. The RRLA further includes a focus on literacy instruction for English language 
learners (ELLs). 

In addition to all coursework in the program, candidates must successfully 
complete a comprehensive exam (comps), which serves as a culminating activity 
for the program. Traditionally, all candidates in M. A. programs in education at this 
campus sit for a three-hour comps exam offered twice a year (during fall and spring 
quarter). During this exam candidates write essay responses to a series of questions 
prepared by faculty within their specific graduate programs. At the beginning of 
the quarter, candidates are provided with a series of practice or study questions. 
Typically, between the time candidates receive the questions and the actual date 
of the exam, they form study groups and work together in preparing responses to 
those questions. They also may ask faculty to clarify questions and review their 
responses before the date of the exam. For the RRLA program, candidates were 
given ten study questions, two of which showed up as mandatory questions on the 
exam. The candidates answered two additional questions, which were chosen from 
the rest of the questions. Each response was scored by a minimum of two faculty 
members, with some faculty scoring responses to more than one question. At times, 
some faculty scored more than 40 responses. 

Although the RRLA faculty rewrote some comps questions each year, those 
faculty who scored the exams began to notice similarities between candidates’ 
responses due to the fact that candidates studied together and often memorized 
similar, if not identical, responses to the study questions. Because of the similarity 
of exams and a general dissatisfaction with the traditional measure of using a sit- 
down written examination to assess our candidates, we looked for more authentic 
and meaningful ways to assess our candidates’ knowledge and skills. 

In fall quarter, 2004, the faculty decided to change the nature of the RRLA 
comprehensive exam in order to more effectively measure candidates’ knowledge 
and skills in conditions where they would be able to demonstrate competence situ- 
ated in the context of working with “real students.” We designed an exam to give 
our candidates different opportunities to examine, understand, and demonstrate 
their knowledge of relationships among theory, research, practice, and decision- 
making. The idea was to simulate a clinical experience, to have the candidate act 
as a reading specialist and demonstrate the knowledge and skills learned in the 
program. In other words, we asked our candidates to “apply and show what they 
know.” We utilized a common six-step model in designing the case method that 
asked candidates to: (1) identify the educational issues involved; (2) think about 
the case from multiple points of view (e.g., parent, student, teacher, principal); (3) 
use professional knowledge (e.g., learning theory) to discuss the case; (4) project 
courses of action that might solve the problem; (5) determine the consequence that 
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might follow from each course of action generated; (6) after evaluating each, choose 
courses of action to be followed and decide how to evaluate the effectiveness of 
the plan (Greenwood, 1996, p. 59). 

Thus, the new comps exam asked candidates to administer a battery of as- 
sessments to one child; analyze the assessment results in terms of reading, writing 
and English language development (ELD); and determine the child’s strengths and 
weaknesses in each area and write an appropriate intervention plan. It is important to 
note that our campus not only serves a diverse population, but teachers who attend 
our programs themselves teach an extremely diverse range of students including 
a considerable number of ELLs, most of whom initially learn to read in English, 
their second language. As a result, our graduates need the knowledge and skills to 
work effectively with ELLs. 

Therefore we asked each candidate to develop his/her own individually written 
case study of a “real” struggling reader. Guidelines required candidates to begin by 
describing the student’s background and then to construct the case study using data 
from the assessments and their knowledge about effective literacy instruction in 
order to make recommendations for the student. Within the case study, candidates 
must demonstrate and apply deep knowledge of curriculum and instructional ap- 
proaches to use with struggling readers, including a broad and in-depth knowledge 
of instructional programs and specialized materials. 

Once the new comps exam was conceptualized, agreed upon, developed, and 
implemented, faculty members were faced with the challenge of evaluating the 
case studies that candidates generated. First, we needed to decide on the rubric that 
would be most effective to score the cases, since the existing department rubric was 
no longer appropriate for the new exam. Because our candidates were no longer 
writing exams by hand in one sitting, we analyzed the issues that scoring the cases 
presented. For example, faculty believed that since students would have more time 
to write and would theoretically submit more polished word-processed exams, the 
new rubric should allow for grammar, spelling, writing style, and use of references 
to be weighted differently in the scoring. 

The authors of this study applied for and received a small assessment grant 
in winter quarter, 2005, to address issues related to evaluating the case studies 
developed by our candidates for the comprehensive exam. We used the grant to 
develop a new scoring rubric (see Figure One), guidelines for scoring and the 
gateway requirements (see Figure Two), which documented the use of current 
research on reading, writing and ELD, APA format, page numbers, and the use of 
www.turnitin.com to scan for plagiarism, etc. The purpose of the gateway was to 
immediately screen out any case studies that had missing requirements prior to the 
faculty evaluators scoring them. Exams that did not meet the gateway requirements 
were not accepted and candidates were given 24 hours to complete the exam. 


129 


Using Case Studies To Assess Candidates’ Knowledge and Skills 


Figure One: RRLA Comprehensive Exam Scoring Rubric. 




Reader # Student CE1D # □ Reading □ Writing □ ELD Score 

Rubric for Reading M.A. and Reading and Language Arts Specialist Credential Comprehensive Exam (Ulanoff, Fingon and Beltran, 2005) 

Scoring: Candidates are scored on three sections: reading, writing, and English language develonment using the following rubric. Candidates must have an average score of 
no less than 2.5. and no sections with a rating below 2 to pass. In general, a “3” rating is considered passing or meeting the requirements. Please check the bullets that were 
key in your decision rating and list comments as necessary. Plus (+) or minus (-) ratings can be given in any section. 

Level 1 — Unacceptable (fail) 

Poor organization; poorly 
structured writing and many errors. 

Limited, inaccurate or irrelevant 
overall information. 

Inaccurate description and/or poor 
interpretation 

Makes significant errors during 
administration and interpretation of 
assessments and plans intervention 
based on erroneous data. 

Demonstrates very limited 
knowledge without consistent 
application. 

Inappropriate or less than 3 
intervention strategies regardless 
of demonstrated needs. 

Inaccurate with limited or no 
details or references. 

Lacks overall mastery. 

No questions or poorly written or 
irrelevant questions. 

Lacks understanding. 


Level 2 — Minimal, needs 
improvement (fail) 

Fair organization and overall 
writing skills with few minor 
errors. 

Accurate and relevant 
information. 

Accurate description and 
limited interpretations. 

Adequately administers and 
interprets required assessments 
and uses results with a few 
minor errors. 

Demonstrates and adequately 
applies basic knowledge. 

Mostly appropriate, but lacks 
full detail. 

Mostly accurate and justified 
with some details and 
somewhat relevant references. 

Minimal knowledge and 
mastery. 

3 — 5 adequately written 
questions. 

Basic understanding. 

Level 3 — Passing 
(Pass) 

Very good or good 
organization, writing 
skills and mechanics. 

Detailed, relevant and 
accurate overall 
information 

Accurate description and 
adequate interpretation. 

Correctly administers 
and interprets required 
assessments and uses 
data appropriately. 

Demonstrates and 
applies good knowledge 

Appropriate. 

Accurate and justified 
with good details, 
including relevant 
references or sources. 

Clear and good mastery . 

5 well-written questions 

Good understanding. 

Rating 4 — Exceptional, 
“Above and Bevond” (pass) 

Excellent organization, writing 
skills and mechanics. 

Concise, detailed, relevant and 
accurate overall information. 

Accurate description and 
interpretation. 

Correctly administers and 
thoroughly interprets required 
assessments (may include 
others) and uses data 
appropriately. 

Demonstrates and effectively 
applies in-depth knowledge. 

Highly appropriate (highest 
priori ty). 

Accurate and justified 
explanations with excellent 
details and at least 3 relevant 
references or sources per area. 

Clear and extensive mastery. 

5 or more insightful and well- 
written questions. 

Excellent understanding. 

Evaluation Criteria 

Format and organization: gateway requirements, 
exam organization, writing skills, and grammar, 
spelling and punctuation 

Provides background information about the 
student: academic, social/emotional/physical, and 
sociological information necessary to describe 
student in terms of reading, writing and ELD. 

Understanding of overall assessment task: 
Description of testing instruments and 
interpretation of results. 

Administration and interpretation of reading, 
writing and ELD assessments and uses results (u 
plan an intervention based on student needs. 

Knowledge of current research and theory and 
connection to practice in reading, writing and 
ELD and application to intervention plan. 

Selects 3 intervention strategies and 
recommendations for reading, writing and ELD 
based on student needs. 

Explanations of test data and details to support 
generalizations about interpretation and 
intervention for reading, writing and ELD. 

Mastery of the professional vocabulary and 
terminology and terminology in reading, writing, 
ELD, and R/LA standards. 

Request for additional information (not 
previously acquired) to assist candidate in 
establishing the best intervention plan based on 
student needs. 

Appropriate usage of intervention plan: 
Understanding of the overall concept of an 
intervention plan. 
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Figure Two. Gateway Requirements (Fingon, Ulanoff & Beltran, 2005). 

Candidates must first meet these minimal overall requirements before final evaluation 
takes place. If all gateway requirements are not met the candidate will be given 24 hours to 
successfully meet these requirements in order for his/her work to be accepted for the final 
evaluation phase. 

3 hard copies of exam 

Cover sheet w / CEID on exam 


CEID only (no name) on all pages 

Maximum 25 pages + appendices 

Pages are numbered 

Typed, double-spaced, 12 pt. font, 

w/ one-inch margins 

Table of contents (w/page numbers) 

organized according to the specified 
sections of the exam 

Please place all 3 copies of the exam + audiocassette, classroom observation form and turnitin. 
com digital receipt copy in a manila envelope identified only by CEID. No other papers will 
be accepted. Please note: file uploaded to www.turnitin.com must be identical to each of the 
3 hard copies (minus the classroom observation form and turnitin.com digital receipt). 

Decision: 

Exam complete/accept as is (please check one): Yes □ No □ 

Exam missing section(s) (see above). Revise and submit by . 

Signature of faculty member accepting exam: . 


.Classroom observation form 
Assessment summary data sheet 
(listing all assessments and results) 
.Samples of required assessments 
and writing samples (in Appendices) 
.Reference page (9 minimum 
required references) 

APA style 

.Copy of turnitin.com receipt 

One audiocassette 
with assessments 


Methodology 

Setting and Sample 

Data were collected in one Charter College of Education (CCOE) at a large, 
diverse, urban state university in Southern California. At the time of the study, the 
CCOE enrolled 15% of the total campus population of 20,000. The student popu- 
lation was approximately 50% Latino, 19% Asian-American or Pacific Islander, 
14% White, Non-Latino, 1.5% African- American, 9% unknown, 5. 5% non-resident 
undocumented and .4% American Indian. Moreover 62.3% of students were women 
and the average age of graduate students was 34. 

The sample for this study consisted of 110 candidates in three cohorts of 
candidates (graduate students) taking the comprehensive exam as a culminating 
activity in the previously described RRLA program, which was the 5 th largest 
graduate program on campus. All candidates held at least a preliminary teaching 
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credential and 92% of them were teaching in K-12 public schools. The rest taught 
in charter and private schools, or served as substitute teachers. 


Method 

Data collection took place from September 2004 to December 2005 and was 
both qualitative and quantitative in nature. All 1 10 candidates completed a com- 
prehensive exam that required them to generate a case study of a struggling reader, 
who was also an ELL. The intention of the exam was to have students demonstrate 
the knowledge and skills related to their future roles as reading specialists in lin- 
guistically and culturally diverse urban settings. The exam was delivered through a 
course-based technology website (WebCT) and candidates were given four weeks 
to complete the exam. All exams were submitted through www.turnitin.com and 
as hard copies. In addition to writing the case study, candidates were required to 
meet a series of gateway requirements (see Figure Two). 

A team of three faculty members (evaluators) scored each exam. The number 
of teams varied each quarter depending on the number of exams and this served 
to stabilize faculty workload related to scoring. On average each evaluator scored 
10-15 exams. Each evaluator in a team read for one of three foci: reading, writing 
or ELD. The evaluators used the 4-point rubric (see Figure One) developed for 
the case study and an overall “3” rating was considered passing or meeting the 
requirements. Candidates needed to receive a mean score of at least 2.5 or better, 
with no score of less than 2.0 in any one section, in order to pass the exam. One 
score below 2 allowed a rewrite of that focus area. It is important to note that 
the same evaluators scored the exams at all three data points. Fall 2004 — Fall 
2005. 


Data Sources 

The qualitative data set consisted of the following: ( 1 ) comprehensive exams 
created and submitted by the candidates; (2) exit survey completed by the candidates 
(see Appendix); (3) rubrics; (4) evaluator comments; and (5) e-mail exchanges 
between candidates and faculty. Quantitative data consisted of exam scores, which 
were analyzed according to focus area. 

Data analysis was based on a review of each data source and coding of data. 
Patterns were identified as they surfaced and these were used to further identify 
salient themes, and categories within and across each data set. Exams were assessed 
based on how well the candidates demonstrated the following: 

• scoring, analyzing, and interpreting multiple sources of authentic data 

related to the case study student; 

• examining and using data in context; 

• translating theory into practice; 
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• engaging in higher order thinking, problem solving, and learning at a 
higher cognitive level; and 

• generating and evaluating possible solutions and making recommenda- 
tions based on results. 

In essence, the evaluators were interested in finding out how candidates “apply 
what they know” and transform what is learned from the experience into practice. 
The major themes that emerged from the data highlighted candidate strengths, the 
link between theory and practice, and candidate weaknesses. These themes fur- 
ther served to identify matches and gaps between program content and candidate 
knowledge and skills. 


Findings 

Overall findings indicate that we were able to use the case studies to measure 
program content knowledge and also to identify the matches and gaps between 
program content and candidate knowledge and skills. A total of 114 candidates 
attempted the exam, but only 1 10 completed the exam; 104 candidates passed the 
exam. In Fall 2004, 95% of candidates passed the exam (42 out of 44) and there 
were 6 rewrites on one focus area. In Spring 2005, 92% candidates passed the exam 
(48 out of 52), and there were a total of 8 rewrites. All 14 candidates (100%) who 
took the exam in fall 2005 passed without the need to rewrite any one area. The 
candidates who failed the exam in Fall 2004 and Spring 2005 received low scores in 
all three focus areas. Table one shows mean scores for all focus areas as well as the 
mean composite score for each cohort, Fall 2004, Spring 2005, and Fall 2005. 

Candidate Strengths 

Candidates who successfully completed the exam demonstrated knowledge 
and skills related to the teaching of reading, writing and ELD as measured by the 
case studies produced for the comprehensive exam. As can be seen from table one, 
there was little difference in scores over the course of the study, with candidates 
in Spring 2005 scoring slightly lower than the candidates in each of the other two 
quarters examined. Ninety percent of exit survey respondents in spring and fall 2005 
indicated that they felt well or extremely well prepared to be reading specialists. 


Table One. Mean Passing Comprehensive Exam Scores, Fall 2004 to Fall 2005 (n=l04). 



Reading 

Writing 

ELD 

Mean 

Composite Score 

Fall 2004 (n=42) 

2.95 

3.08 

3.17 

3.07 

Spring 2005 (n=48) 

2.87 

2.92 

2.62 

2.80 

Fall 2005 (n=14) 

2.96 

3.14 

3.18 

3.09 
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Upon analyzing the exams, rubrics, and evaluator comments from three candidate 
cohorts over a two-year period, we found that there was greater specificity in the 
instructions given to the candidates, the requirements of the exam, and faculty 
expectations for procedure and content. By Spring 2005, both evaluators and 
candidates more clearly understood the content of the exam as well as what was 
required in terms of collection and analysis of the case study student data, making 
recommendations for student practice, and linking analysis and recommendations 
to research in the field. 

Facilitating candidate strengths with the gateway and the four-point rubric : 
As we transitioned from the old ten-point rubric to the new four-point rubric for 
evaluation of the case studies, initially there was a lack of understanding and 
consistency among faculty about interpreting and using the rubric. The gateway 
requirements (Figure Two) were created to spell out the minimal requirements in 
terms of acceptable content and format for candidates. Since only exams that met 
the gateway requirements were accepted, the evaluators scored only complete ex- 
ams. In essence, the gateway requirements helped to explain exam procedures for 
candidates as well as faculty who served as evaluators. While the gateway require- 
ments clarified procedural issues (e.g., how many pages, the use of APA format, 
the use of the id number, the number of references, etc.) the 4-point scoring rubric 
clarified faculty expectations regarding the content of the exam. 

It is important to note that the RRLA program coordinator, one of the authors of 
this paper, oversaw the exam process. She met with students to review expectations for 
the case study, the online delivery and submission systems, and served as the contact 
person for support during the exam. The coordinator organized the blind review of 
exams, including the assignment of exams to faculty for scoring, and determined 
the timeline for the exam. She also compiled scores and notified students as to their 
pass/fail status. The same program coordinator served throughout the study. 

Examining content for demonstration of candidate strengths'. We found that 
we were able to use the case studies to help identity candidate strengths in terms 
of the content knowledge and skills of our candidates. The candidates consistently 
provided evidence that they understood how to assess students and use assessment 
results to make recommendations for effective instruction, and as noted, scores were 
fairly stable over the course of the study. For example, one of the requirements in 
the case study was to identify strategies that best met the needs of their student for 
remediation in the areas of reading, writing, and ELD, based on their interpretation 
of assessment results. Table two lists the specific intervention strategies used in 
recommendations for the case study students in Fall 2005. These strategies reflect 
the application and understanding of course content and, in most cases, were used 
appropriately in the context of the case study. 
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Table Two. Spring 2005 Comps Case Study Student Strategies. 


Reading 

Writing 

ELD 

Directed Listening Thinking 
Activity (DLTA)* 

Graphic 

organizers* 

Read alouds 

Directed Reading Thinking 
Activity (DRTA)* 

Writers 

workshop* 

DRTA 

Story mapping 

Dialogue journals 

Scaffolding* 

Language Experience Approach 
(LEA) 

Interactive 

writing* 

K-W-L (What you know, 
what you want to know, 
what you learned)* 
or K-W-L Plus 

Vocabulary Self Selection (VSS)* 

Shared writing* 

Writer’s workshop 

Guided reading* 

Author study 

Daily reading 

Directed Reading Activity (DRA) 

Scaffolding 

Contextual redefinition* 

Literature Circles 

Conferencing 

VSS 

ReQuest 

LEA 

Guided reading 

Question- Answer- Relationship 
(QAR) 

Story mapping 

Retelling* 

Story board 

Word walls 

Grand conversations 

Text structure 

Word building 

Multicultural literature 

Reader’s Theatre 

Learning logs 

Sheltered English approach 

Retelling 

Peer editing 

Semantic feature analysis 

Imaging 

Webbing 

Show and tell 

Reciprocal teaching 

Word sorts 

LEA 

Independent reading 



Semantic mapping 



Cloze 



Repeated reading 




Overlap of some strategies occur in more than one area 
* = most frequently described as a strategy by a student in the case study 
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The Link Between Theory and Practice 

Candidates were required to link theory to practice, using published research 
to support their analyses and recommendations, which was demonstrated in their 
analysis of the data from their case study and the recommendations for practice, 
including the strategies listed in table two. The strategies that they chose were 
generally linked to research presented in their coursework. By and large, the case 
studies provided candidates with opportunities to demonstrate their knowledge of 
the professional research related to reading, writing and ELD. 

We analyzed all case studies described in this paper and used the following 
excerpts to illuminate how the case studies enabled candidates to demonstrate their 
ability to link reading, writing, and ELD theory and practice for culturally and 
linguistically diverse urban students. We thought it best to provide excerpts from 
one case study that received a “3,” or passing rating, in the areas of reading, writ- 
ing and ELD to provide a glimpse into a complete case study. All of the examples 
come from one female candidate’s case study of an 8-year-old boy named Jose (a 
pseudonym), who was born in Mexico and came to the United States in 2004. The 
exam was completed in Spring 2005 when Jose was in the 3 rd grade. 

Reading-. In this excerpt the candidate determined that one of Jose’s reading 
needs was in the area of word attack skills. The candidate described and prioritized 
Jose’s needs based on an Individualized Reading Inventory (IRI) and gave specific 
examples of miscues in another portion of the case study. The candidate then provided 
relevant theoretical research sources to support the recommendations she suggested 
for Jose when she offered another approach to learning word attack skills. 

A need displayed by Jose is in the area of word attack skills and many of his miscues 
on the IRI were substitutions for other words with the same initial sound such as 
“laughed" for “left". Although he is in a systematic phonics program there are 
several other strategies available for decoding and learning unknown words. For 
example, Cooper (2003 ) suggests structural analysis because it requires the reader 
to look at the word for meaningful units or parts in order to decode the word or 
figure out its meaning. Another area of need which was made evident through 
the CELDT [California English Language Development Test], IRI, and Slosson 
[Slosson Intelligence Test] was reading comprehension.... 

Writing : In this excerpt on writing the candidate made recommendations 
regarding Jose’s need to develop a good attitude and willingness to write and the 
need for good models to inspire his writing. 

There has to be enthusiasm for writing. That means a teacher who continually 
“sells " writing to his or her students, who knows compelling arguments for the 
importance in writing in today’s society and who makes an event of most writing 
occasions from writing (Walshe, 1979). According to Walshe and others reading is 
inextricably linked to writing, just as talk of adults is linked to infant s acquisition of 
speech. Jose would benefit from daily mini-lessons that model writing skills.... 


136 


Sharon H. Ulanoff, Joan C. Fingon, & Dolores Beltran 


ELD\ In this third excerpt the candidate referred to Jose ’s ELD needs by referencing 
his emerging language skills since his recent arrival in the United States (mentioned 
previously) and the need to increase his overall English language learning: 

Although Jose has developed Basic Interpersonal Communications Skills (BICS) 
he has not acquired academic language and does not have “language proficiency 
or ability to use a language effectively and appropriately throughout the range 
of social, personal, school, work situations required for daily living in a given 
society ” (Peregov & Boyle, 2001, p. 29). His vocabulary can be improved by pro- 
viding additional scaffolding to his lessons with comprehensible input. Interactive 
opportunities between Jose and other students will also allow him to strengthen 
his English and acquire additional BICS and eventually Cognitive/Academic 
Language Proficiency (CALP). 

The candidate concluded this section of the case study by linking many of 
Jose’s areas of need (reading comprehension, word attack skills, and vocabulary) 
with ELD and documenting these suggestions supported in the literature: 

Transferring prior knowledge, translating passages to his native language and 
reflecting on the text in his native language has the potential for improving his 
reading comprehension (Jimenez, 1997). The teacher should tty to learn about 
things that are of more interest to Jose and perhaps call on him for his expertise 
during appropriate class discussions. It is also important to include multicultural 
children ’s literature, which is representative of his Mexican culture... to develop 
a sense of belonging. 

Requesting additional information : A final section, recently added to the case 
study, asks candidates for additional information they wished they had known regard- 
ing their student (since there were strict time limitations on the exam). They were 
asked to elaborate on whom they might ask and to provide a rationale. In this final 
excerpt, the candidate demonstrated her knowledge and skills beyond the confines 
of student and assessor/reading specialist, allowing an indication of understanding 
within a broader school community context. The candidate responded as follows: 

If I had had additional time to work with this student and this project I would have 
liked to have interviewed Jose ’s current teacher and asked Jose s study habits and 
what type of vocabulary instruction was currently being provided? I would have 
liked to know more about his interests and performed an interest inventory, what 
types of books he liked what sort of things he does in his spare time? I would 
have liked to have known his grades from Mexico or had spoken to his previous 
teacher to get more background information about Jose. I would have also liked 
to have observed Jose more in his classroom to get a more realistic picture of 
who he is in class. . . and performed other assessments in order to help Jose . . . this 
information could also have been beneficial in individualizing an instructional 
plan to fit Jose s needs. 
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Candidate Weaknesses 

Post exam survey responses indicated that candidates felt that the exam was 
a measure of the program content — 98% in Spring 2005 and 100% in Fall 2005 
responded that they either agreed or strongly agreed that the content of the exam 
was representative of the program content. However, candidates stated that they felt 
more confident about content related to reading than either writing or ELD based 
on program courses. 

Examination of the data from all 1 1 0 candidates who successfully completed the 
exam between Fall 2004 and Fall 2005 demonstrated candidate weaknesses in linking 
theory to practice in the area of ELD. Candidates specifically struggled with identify- 
ing ways in which to scaffold instruction for ELLs, arguing that “there was only one 
ELD class and I didn’t think that there was enough emphasis on ELD in each of the 
other classes.” Since the RRLA program is designed to infuse content related to the 
assessment and instruction of ELLs throughout the program, comments such as this 
one serve to highlight the need for further program examination. This finding mir- 
rors a recent report from the California State University Chancellor’s Office, and this 
content is especially important in relation to the preparation of our RRLA candidates 
since they teach in classrooms with high numbers of ELLs and need knowledge and 
skills related to ELD in order to effectively serve their students. 

Anotherpattern of candidate weakness identifiedby the exams, post exam survey 
data, and evaluator comments was found in the administration and interpretation of 
writing analyses. A third of the faculty who evaluated the exams complained about 
the limitations of required writing assessments, citing candidates’ lack of proficiency 
in this area. This pattern of weakness generated ongoing debate about how best to 
teach and assess candidate knowledge and skills related to teaching writing. One 
candidate’s post exam survey response stated, “I would have preferred to have some 
[more] experience using the [required] vocabulary index and the t-unit analysis 
[writing assessments].” Faculty felt that further examination of these data would be 
helpful in program evaluation for ongoing program revision, including the revision of 
course content that addresses knowledge and skills aligned to program standards. 


Conclusions and Implications 

Moje and Wade (1997) argue that case-based discussions in general can help 
teachers to challenge preconceived notions about teaching and learning as they explore 
the relationships between “. . .knowledge, ability and literacy” (p. 705). As teachers 
move beyond discussions to examine student artifacts (including assessment results) 
as direct records of the classroom experience, these artifacts support the development 
of a case as an interpretive account of the subject’s strengths and weaknesses (Shul- 
man & Kepner, 1999). As the teachers in the RRLA program examined the strengths 
and weakness of their case study students, we look at the cases they developed to 
measure their knowledge and skills as reading specialists. 
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While some research has been done in the area of case-based assessment, 
limited practice exists using candidate-generated case studies as assessment to 
measure student outcomes in graduate programs. Upon close examination of these 
case studies, we see how they demonstrate the ways in which candidates process 
information and apply it to “real life” situations. Most importantly, our study dem- 
onstrates the multiple uses of candidate-generated case studies for assessment as 
they provide evidence of individuals’ knowledge and skills, and of the knowledge 
and skills of the group as a whole. They further afford us a glimpse into overall 
program effectiveness. 

As faculty researchers, we have begun to see more benefits from doing assess- 
ment consciously and conscientiously. The three authors of this paper have operated 
as a small team, systematically examining candidates’ work. Our collaboration has 
led us to new insights and the particular action of improving the RRLA program. 
We have come to view the graduate case studies as part of a non-linear, recursive 
data analysis cycle that is process of noticing, collecting and thinking (Seidel, 1 998, 
p. 2). Because our data analysis cycle builds on existing understanding and action 
to create new understanding and action, with each iteration we notice new things 
in the data and react. As we have moved through our process and gained insight 
into the potential of our graduate case studies, we have returned frequently to our 
earlier sources to gather new data for improving our course content. Because the 
case studies are authentic applications contextualized within simulated practice 
with real students, the varied competencies developed in our program become 
transparent for program faculty. Like our candidates in the analysis of their own 
assessment data, we have access to a more complex set of data than was previously 
available, allowing us to examine the RRLA program more effectively. 

Central to our work is the importance of looking more closely at candidates’ 
original work that was produced in the case studies. In essence, our focus as a team 
examining a culminating activity (the comprehensive exam) shifted from more forma- 
tive assessment of our course content to summative assessment of program content. 
This shift of focus was also noticeable in the faculty who scored the cases, as the 
matches and gaps between the exam and program content became more transparent. 
While an external or outside evaluator might have been useful for investigating program 
outcomes, what emerged from our study was the importance of faculty examining 
authentic work samples as one means of measuring program outcomes, specifically 
the ways in which the RRLA candidates were able to demonstrate understanding of 
program content and its application to “real life” situations. 

Some of the unexpected outcomes of this study for faculty included a better 
understanding of the overall graduate program. With the inception of the use of the 
case studies, faculty meetings and conversations were much more centered on the 
interrelationship between course requirements, program goals, and our candidates’ 
overall abilities to write about and reference what they know. Moreover, the RRLA 
program standards allowed us to examine more closely which courses might cause 
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information gaps or overlaps for students in applying their coursework to the prepara- 
tion of the case study. Our new understandings led us to launch a collaborative effort 
to ensure a stronger alignment of standards across courses. Close examination of 
the case studies also gave faculty more insight into each candidate’s understanding 
of linkages between theory and practice in the areas of reading, writing, and ELD. 
Over the course of the refinement of both the process and content of the exam, we 
have regularly shared our findings with program faculty in formal and informal 
forums. As a result of these focused conversations, faculty members have engaged 
in more meaningful dialogue about desired expectations and how course features 
and assignments support both formative and summative assessment. 

Other positive outcomes that came out of this study included the success of 
the new gateway requirement and rubric in establishing more consistency in the 
scoring process both across cases and among evaluators. Additional interesting 
evidence showed that while candidates were responding to our criteria, they also 
had issues and concerns related to course and program standards and expectations. 
Clearly, we have just begun to understand the value of the case studies and what 
other powerful messages they offer. 

Our inquiry involved us in a dynamic, ongoing process and the result has been 
the emerging social construction of a “learning culture” (Shepard, 2004). While our 
study is limited in scope, the findings offer an interesting and different perspective 
on uses of cases studies, specifically candidate-written cases studies for assessment 
purposes. In addition, the study’s outcomes seem timely and relevant in contributing 
to the growing body of research that supports alternative assessment methods. 
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Appendix 

EDCI 596 Student Comprehensive Exam Exit Survey 
(© Fingon , Ulanoff, & Beltran, 2004) 

Quarter entered program 

1 . Describe your current position. 

2. What is your approximate GPA? 

Please rate each of the following on a 1-5 scale, where (1) is “Strongly Disagree,” (2) is 
“Disagree,” (3) is “No opinion/neutral,” (4) is “Agree,” and (5) is “Strongly Agree.” 


3. The case study tasks and questions were representative of the content of the MA in the 
RRLA program. 


1 

2 

3 

4 

5 

Strongly 

Disagree 

Disagree 

No-Opinion/ 

neutral 

Agree 

Strongly 

agree 
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4. The support I received regarding my questions about the exam was helpful to the process. 


1 

2 

3 

4 

5 

Strongly 

Disagree 

Disagree 

No-Opinion/ 

neutral 

Agree 

Strongly 

agree 


5. I enjoyed the take-home format of the exam. 


i 

2 

3 

4 

5 

Strongly 

Disagree 

Disagree 

No-Opinion/ 

neutral 

Agree 

Strongly 

agree 


6. Which of the following were helpful to you during the exam? Check all that apply?* 


□ The comps meeting 

□ The exam instructions 

□ The comps WebCT 

□ The exam “test pilot” 

□ The gateway requirements 

□ The tech support 

□ The scoring rubric 

□ The exam timeline 


7. Please list any challenges that you had taking the exam. 

8. Please list any positive outcomes you experienced during the exam.* 

9. Do you have any suggestions for improving the exam? 

10. How well do you feel prepared to be a Reading Specialist? 


1 

2 

3 

4 

5 

Unprepared 

Minimally 

Prepared 

Somewhat 

Prepared 

Well-prepared 

Extremely 

Well-prepared 


1 1 . How likely do you think it is that you will become a reading specialist in the near 
future? 


Don’t Know 

Unlikely 

Somewhat Likely 

Likely 

Highly Likely 


* Questions 6 and 8 were added to the survey in Fall 2005. 
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